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VIDEO SIGNAL PROCESSING 

This invention relates to video signal processing and is particularly 
concerned with non-linear filtering. 
5 It has been found that in a wide variety of video signal processes - 

including de-interlacing, decoding, enhancement, noise reduction, and 
standards conversion - considerable advantage can be secured by the use 
of complex non-linear filters. It has been found in particular that polynomial 
filters can be very useful. In many applications, quadratic behaviour in the 
0 filter is not sufficient and third or higher orders are typically necessary. 
Where real time operation is required, hardware implementations are 
usually essential and the hardware costs of such high order polynomial 
filters are substantial. 

It is an object of the present invention to provide improved methods 
5 and apparatus in video signal processing which offer third or higher order 
behaviour in a relatively simple filter architecture. 

Accordingly, the present invention consists, in one aspect, in a 
method of video signal processing in which the outputs of at least three 
linear filters are multiplied together. 

Preferably, the product of the outputs of two filters is formed and a 
linear filtering operation conducted on that product, prior to multiplication by 
the output of the third filter output. 

In one form of the invention, linear and non-linear filtering operations 
are conducted in parallel with the results of said operations being combined 
to produce an output; the non-linear filter operation comprising the 
multiplication together of the outputs of at least three linear filters. 

In another aspect, the present invention consists in apparatus for 
video signal processing comprising an input terminal for receiving an input 
video signal; first, second and third linear filters each connected with the 
input terminal; a first multiplier for multiplying together the respective 
outputs of the first and second filters; and a second multiplier for multiplying 
together the respective outputs of the first multiplier and the third filter. 



Advantageously, a filter is interposed between the output of the first 
multiplier and the second multiplier. 

Preferably, the apparatus further comprises a linear filter path 
connected with the input terminal, and a combiner for combining the outputs 
of the linear filter path with the output of said second multiplier. 

Suitably, a filter is interposed between the output of the second 
multiplier and said combiner. 

The invention will now be described by way of example with 
reference to the accompanying drawings in which: 

Figure 1 is a block diagram of video signal processing apparatus 
according to the invention, in the form of a vertical de-interlacing 
filter; 

and 

Figure 2 is a diagram similar to Figure 1, illustrating a modification. 

The embodiment of a vertical de-interlacer is used to exemplify the 
present invention for reasons of simplicity. Better results can be obtained 
by also using horizontal and/or temporal information in ways which will be 
evident. The filter could equally well be applied to the other problems in 
video processing. 

In Figure 1 , the new architecture can be seen to consist of two signal 
paths. The linear signal path contains a traditional, vertical, six tap, linear 
filter (h ljn ) which has a typical sin(x)/x structure. If this were to be used 
without the non-linear signal path it would produce reasonable pictures, but 
they would contain some artefacts due to the interpolation process, notably 
jagging on diagonal and curved edges. 

In the non-linear signal path, the output of two four point linear filters 
(h^ and h 2 ) are multiplied together and passed through a two point linear 
filter (h 4 ). The output of this is then multiplied with the output of a five point 
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linear filter (h 5 ). Finally the resulting signal is filtered through another two 
point linear filter (h 6 ) before being added on to the linear path. Although in 
this case the filter lengths are 4, 5 and 2, larger filters with more taps can 
be used to give better results. The lengths (or more generally, the sizes) of 
5 the filters need not be related and can be made larger or smaller to provide 
different trade-offs between quality and cost. 

The filter coefficients are selected by 'training' the filter on real 
pictures. In this example of de-interlacing a still frame is taken and split 
into fields. A set of coefficients is used to estimate Field 2 from Field 1 and 

10 the mean squared error between the estimate of Field 2 and the original 
Field 2 is measured. A genetic algorithm can then be used to search the 
multi-dimensional filter space for the set of filter coefficients that gives the 
lowest mean squared error. 

If this non-linear de-interlacer is tried on the EBU/SMPTE test picture 

15 "Girl with Toys", the non-linear path is found to reduce the average mean 
squared error by approximately 15% with respect to the linear filter. There 
is also a noticeable reduction in jagging. A polynomial filter with the same 
number of input pixel taps produces an almost equivalent reduction in error. 
A major advantage of this new architecture over the polynomial filter 

20 can be seen by considering the number of multiplications of pixels; 
multiplications of pixels by a constant; and additions, that each filter 
requires. These are shown in Table 1. 





Polynomial 
Filter 


New 
Architecture 


Multiplication 


50 


2 


Multiplication by 
a constant 


34 


23 


Additions 


34 


24 



Table 1 : Comparison of complexity of filters 



It can be seen that the largest reduction is in the multiplication of 

) 

pixels. This is particularly significant as these are the most expensive to 
implement. 

In summary, the new architecture is able to reduce many of the 
5 artefacts associated with traditional linear interpolation whilst being relatively 
simple to implement. 

Figure 2 illustrates a modification in which the architecture is 
simplified through omission of the filters h 4 and h 6 . In other words, the 
direct product is formed of the outputs of filters h 1t h 2 and h 3 without 

10 intervening filtering of the product of the outputs of filters h, and h 2 . This 
may under some circumstances produce less ideal filter behaviour, but the 
reduction in hardware complexity will often more than compensate. A 
particular advantage is that the three remaining filters can all make use of 
the same memory architecture. 

15 It should be understood that this invention has been described by 

way of example only and that a wide variety of modifications are possible 
. without departing from the scope of the invention. Thus, whilst the 
separation into linear and non-linear paths offers important advantages, 
such as the option to preserve higher bit accuracy in the linear path, it will 

20 not always appropriate. Similarly, the described use of vertical filters is - as 
has been explained - merely an example. Horizontal, vertical and temporal 
filters can be employed and filters can have one, two or three of these 
dimensions. Whilst, FIR filters will be important, the invention also 
encompasses other forms of linear filter such as recursive filters. The filters 

25 which are to be multiplied together need not be of the same category. 



A method of video signal processing in which the outputs of at least 
three linear filters are multiplied together. 

A method according to Claim 1, wherein the product of the outputs of 
two filters is formed and a linear filtering operation conducted on that 
product, prior to multiplication by the output of the third filter output. 

A method according to Claim 1 or Claim 2, wherein linear and non- 
linear filtering operations are conducted in parallel, with the results of 
said operations being combined to produce an output; the non-linear 
filter operation comprising the multiplication together of the outputs of 
at least three linear filters. 

Apparatus for video signal processing comprising an input terminal 
for receiving an input video signal; first, second and third linear filters 
each connected with the input terminal; a first multiplier for 
multiplying together the respective outputs of the first and second 
filters; and a second multiplier for multiplying together the respective 
outputs of the first multiplier and the third filter. 

Apparatus according to Claim 4, wherein a filter is interposed 
between the output of the first multiplier and the second multiplier. 

Apparatus according to Claim 4 or Claim 5, wherein the apparatus 
further comprises a linear filter path connected with the input 
terminal, and a combiner for combining the outputs of the linear filter 
path with the output of said second multiplier. 

Apparatus according to Claim 6, wherein a filter is interposed 
between the output of the second multiplier and said combiner. 
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Figure 2 



